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BUFFER DESCRIPTOR DATA STRUCTURE FOR COMMUNICATION LINK 
BETWEEN DECODE AND DISPLAY PROCESSES IN MPEG DECODERS 

RELATED APPLICATIONS 

[0001] This application claims priority to Provisional 
Application for U.S. Patent, App. Ser. No. 60/441,820, 
filed January 21, 2003 by Sandeep Bhatia, entitled "Buffer 
Descriptor Data Structures for Communication Link between 
Decode and Display Processes in MPEG Decoders", which is 
incorporated herein by reference for all purposes. 

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 

[0002] [Not Applicable] 

[MICROFICHE/COPYRIGHT REFERENCE] 

[0003] [Not Applicable] 

BACKGROUND OF THE INVENTION 

[0004] This application is related to digital video data, and 
more particularly to buffer descriptor data structures 
between the decode and display processes in a decoder. 

[0005] Video decoders decode a video bit -stream encoded 
according to a predetermined standard syntax, such as MPEG- 
2 or Advanced Video Compression (AVC) . An encoder 
generating a compressed video bit-stream makes a number of 
choices for converting the video stream into a compressed 
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video bit-stream that satisfies the quality of service and 
bit -rate requirements of a channel and media. However, 
decoders have limited choices while decoding the compressed 
bit stream. The decoder uses the decisions made by the 
encoder to decode and present pictures at the output screen 
with the correct frame rate at the correct times, and the 
correct spatial resolution. 

[0006] Decoding can be partitioned in two processes - the 
decode process and the display process. The decode process 
parses through the incoming bit stream and decodes the bit 
stream to produce decode images which contain raw pixel 
data. The display process displays the decoded images onto 
an output screen at the proper time and at the correct and 
appropriate spatial and temporal resolutions as indicated 
in the display parameters received with the stream. 

[0007] Some encoders, such as MPEG encoders, encode the time 
at which each decoded image is to be presented along with 
the compressed pixel data for the image. The spatial 
resolution of a picture, such as horizontal size and 
vertical size is also encoded along with the compressed 
pixel data of a picture. At the start of every sequence, 
the frame rate (temporal resolution) at which encoded 
images in the sequence are to be output onto the screen is 
indicated in the compressed bit stream. The decode process 
parses all of the foregoing parameters while decoding the 
compressed bit stream. While some of the information parsed 
during the decode process, such as horizontal and vertical 
picture sizes, are important only for the decode process, 
other parameters such as the presentation time and aspect 
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ratio of the picture are to be used exclusively while 
displaying the picture. For example, MPEG-2 uses 
presentation time stamps to indicate the presentation time 
for an image on the display. In order for the display 
process to accomplish its objective of being able to 
present image buffers at their correct intended 
presentation time, the display process uses various 
parameters, including the presentation time stamp, parsed 
during the decode process. In some standards, such as MPEG- 
2 and AVC, the number of parameters can be quite large. 
Additionally, the decode process is also responsible for 
altering the configuration of the display hardware 
Alternatively, decode process is also responsible for 
detecting if a reconfiguration of display hardware will be 
necessary and making necessary preparations for it. if the 
display parameters such as displayable horizontal and 
vertical size, aspect ratio change before the pictures with 
changed parameters are processed by the display hardware. 

[0008] Further limitations and disadvantages of conventional 
and traditional approaches will become apparent to one of 
skill in the art through comparison of such systems with 
embodiments presented in the remainder of the present 
application with reference to the drawings. 
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BRIEF SUMMARY OF THE INVENTION 

[0009] A system, method, and apparatus for providing display 
parameters from the decode process to the display process 
are presented herein. The decode process receives images 
which are encoded according to a predetermined standard. 
Included with the encoded images are parameters which 
facilitate the decode and display processes. The decode 
process decodes the encoded images as well as the 
parameters and stores each image in a separate image 
buffer. Additionally, the decode process stores the 
parameters which facilitate the display process in a buffer 
descriptor structure associated with the image buffer. The 
display process uses the parameters stored in the buffer 
descriptor structure during the display process. 

[0010] These and other advantages and novel features of the 
embodiments in the present application will be more fully 
understood from the following description and in connection 
with the drawings . 
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BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS 

[0011] FIGURE 1 is a flow diagram describing a decode and 
display process in accordance with an embodiment of the 
present invention; 

[0012] FIGURE 2 is a block diagram describing an exemplary 
decoder system for decoding and displaying images; 

[0013] FIGURE 3 is a block diagram describing an exemplary 
MPEG video sequence; 

[0014] FIGURE 4 is a block diagram describing an exemplary 
MPEG decoder configured in accordance with an embodiment of 
the present invention; and 

[0015] FIGURE 5 is a flow diagram describing the operation of 
the MPEG decoder in accordance with an embodiment of the 
present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 



[0016] Referring now to FIGURE 1, there is illustrated a 
block diagram of an exemplary decoder 100 for displaying 
images. The decoder receives encoded data 105 that includes 
encoded images 105a and associated parameters 105b and 
displays the images on the display device 110. An encoder 
encodes the images according to a predetermined standard. 
The predetermined standard can include, for example, but is 
not limited to, MPEG, MPEG-2, or AVC. The encoder also 
encodes a number of parameters for each image that 
facilitate the decoding and displaying process. These 
parameters can include, for example, the decode time, 
presentation time, horizontal size, vertical size, or the 
frame rate. The encoder makes a number of choices for 
encoding the images and parameters in a manner that 
satisfies the quality requirements and channel 
characteristics. However, the decoder 100 has limited 
choices while decoding and displaying the images. The 
decoder 100 uses the decisions made by the encoder to 
decode and display pictures with the correct frame rate at 
the correct times, and the correct spatial resolution. 

[0017] The decoder can be functionally partitioned into two 
sections - a decode engine 115 and a display engine 120. 
The decode engine 115 decodes the encoded images and 
parameters and generates decoded images . Decoding by the 
decode engine 115 can also include decompressing compressed 
images, wherein the images are compressed. The decoded 
images include the raw pixel data. Additionally, the decode 
engine 115 decodes the parameters associated with the 
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images. The display engine 12 0 takes care of displaying the 
decoded images onto the display device 110 at the proper 
time and at the correct spatial and temporal resolution. 

[0018] The display engine 12 0 lags behind the decode engine 
115 by a delay time. In some cases, the delay time is not 
constant. For example, in MPEG-2 and AVC, the order the 
images are decoded and the order the images are displayed 
are not necessarily the same. Where the display engine 120 
lags behind the decode engine 115, the decoded images are 
buffered in image buffers 125a for display by the display 
engine 120. In order for the display engine 120 to 
accomplish its objective of being able to present the 
decoded images at their correct intended presentation time, 
the display engine 12 0 needs various parameters decoded by 
the decode engine. Accordingly, the decode engine 115 
stores the parameters needed for displaying the images in a 
data structure 12 5b. There is a one to one correspondence 
between data structures 125a and 125b. The display engine 
120 retrieves the parameters needed for displaying the 
images on the display device 110 and displays the images on 
the display device 110 based on the parameters. 

[0019] Referring now to FIGURE 2, there is illustrated a flow 
diagram describing the decoding and displaying of an image 
in accordance with an embodiment of the present invention. 
At 205, data comprising encoded images and encoded 
parameters is received by a decode engine. At 210, the 
decode engine decodes the image and parameters . The decoded 
image is buffered in an image buffer (at 215) and the 
parameters are stored in a parameter buffer (at 22 0). 
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associated with the image buffer. At 225, the display 
engine retrieves the parameters stored in the buffer. The 
display engine uses the parameters retrieved during 225 to 
display the decoded image (23 0) . 

[0020] Referring now to FIGURE 3, there is illustrated a 
block diagram describing an exemplary MPEG- 2 video 
sequence. Pursuant to MPEG-2, digital frames are divided 
into 16x16 pixel portions represented by a data structure 
known as a macroblock 305. The macroblock represents 
encoded 16x16 pixels, taking advantage of both spatial 
and/or temporal redundancy. 

[0021] The macroblocks of a frame are grouped into macroblock 
groups known as slices or slice groups. The slice is 
represented by a data structure known as a slice layer 310. 
The slice layer 310 comprises each of the macroblocks 305 
grouped into the slice in addition to other parameters 312. 
The other parameters include quantization values 312a and 
an address associated with the slice 312b. 

[0022] Each of the slice layers 310 associated with an image 
are grouped into a data structure known as a picture, or 
picture layer 315. The picture includes each of the slice 
layers 310 representing the slice groups forming the image. 
Additionally, the picture includes additional parameters 
350. The parameters can include, for example, picture 
structure indicator 350a, progressive flag 350b, a 
presentation time stamp (PTS) present flag 350c, a 
progressive frame flag 350d, a picture structure indicator 
350e, a PTS 350f, and pan-scan vectors 350g, aspect ratio 
350h, decode and display horizontal size parameter 350i, a 



8 



decode and display vertical size parameter 350j , a top 
field first parameter 350k, and a repeat first field 
parameter 3501. It is noted that in the MPEG-2 standard, 
additional parameters may be included. However, for 
purposes of clarity, some parameters are not illustrated in 
FIGURE 3 . 

[0023] Other parameters are a function of the parameters. For 
example, the Still Picture Interpolation Mode (SPIM) is a 
function of the picture structure indicator 350a and the 
progressive flag 350b. The SPIM represents the display 
interpolation mode to be used for a still picture and 
Personal Video Recording (PVR) application such as slow 
motion when real time decode is turned off. The SPIM 350a 
controls the way a static frame picture can be displayed 
onto a screen, for example when user wishes to pause on a 
certain frame or when the encoders encode the presentation 
time stamps of pictures in stream in such a way that 
decoders are forced to display one frame repetitively. 
These actions can include displaying the last field, 
displaying the last displayed top and bottom field pair, 
and sending out all frames lines as both top and bottom 
fields. The amount of motion between two fields of a frame 
determines which SPIM mode gives the best visual quality. 

[0024] As another example, the motion picture interpolation 
mode (MPIM) 350b is also function of the picture structure 
indicator 350a and the progressive flag 350b. The MPIM is a 
one-bit value used while displaying moving pictures. If 
the bit is set, then a complete progressive frame isoutput 
onto screen instead of breaking it into top and bottom 
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fields. If the bit is reset, then the top or bottom field 
is sent depending on if the display hardware requires the 
top or bottom field. 

[0025] The progressive frame parameter 350d indicates whether 
the picture has been encoded as a progressive frame. If 
the bit is set, the picture has been encoded as a 
progressive frame. If the bit is not set, the picture has 
been encoded as an interlaced frame. 

[0026] The picture structure parameter 350e specifies the 
picture structure corresponding to the image buffer. Pan 
scan vectors 350g specify the displayable part of the 
picture. The aspect ratio 350h indicates the aspect ratio 
of the image buffer. The decode and display horizontal size 
parameters 350i indicate the decoded and the displayable 
horizontal sizes of the image buffer, respectively. 

[0027] The top field first parameter 350k is a one-bit 
parameter that indicates for an interlaced sequence whether 
the top field should be displayed first or the bottom field 
should be displayed first. When set, the top field is 
displayed first, while when cleared, the bottom field is 
displayed first. 

[0028] The repeat first field 3501 is a one-bit parameter 
that specifies whether the first displayed field of the 
picture is to be redisplayed after the second field, for an 
interlaced sequence. For progressive sequence, the repeat 
first field 3501 forms a two-bit binary number along with 
the top field first parameter 350k specifying the number of 
times that a progressive frame should be displayed. 
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[0029] The picture layers 315 representing the images of a 
video are grouped together in a data structure known as the 
sequence layer 355. The sequence layer 355 also includes 
additional sequence parameters 360. The sequence parameters 
can include, for example, a progressive sequence parameter 
360a, and a frame rate parameter 360b. 

[0030] It is noted that in MPEG-2 standard, additional 
parameters may be included. However, for purposes of 
clarity, some parameters are not illustrated in FIGURE 3. 

[0031] The progressive sequence parameter 360a is a one-bit 
parameter that indicates whether the video sequence 360 has 
only progressive pictures. If the video sequence 360 
includes only progressive pictures, the progressive 
sequence parameter 3 60a is set. Otherwise, the progressive 
sequence parameter 360a is cleared. The frame rate 360b 
indicates the frame rate of the video sequence. 

[0032] 

[0033] The video sequence 360 is then packetized into a 
packetized elementary stream and converted to a transport 
stream that is provided to a decoder. 

[0034] Referring now to FIGURE 4, there is illustrated a block 
diagram of a decoder configured in accordance with certain 
aspects of the present invention. A processor, that may 
include a CPU 490, reads an MPEG transport stream into a 
transport stream buffer 432 within an SDRAM 430. The data is 
output from the transport stream presentation buffer 432 and 
is then passed to a data transport processor 435. The data 
transport processor then demultiplexes the MPEG transport 
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stream into its preliminary elementary stream constituents 
and passes the audio transport stream to an audio decoder 
460 and the video transport stream to a video transport 
processor 440. The video transport processor 440 converts 
the video transport stream into a video elementary stream 
and transports the video elementary stream to an MPEG video 
decoder 445. The video elementary stream includes encoded 
compressed images and parameters. The MPEG video decoder 445 
decodes the video elementary stream. The MPEG video decoder 
445 decodes the encoded compressed images and parameters in 
the video elementary stream, thereby generating decoded 
images containing raw pixel data. 

[0035] The display engine 450 is responsible for and operable 
to scale the video picture, render the graphics, and 
construct the complete display among other functions. Once 
the display is ready to be presented, it is passed to a 
video encoder 4 55 where it is converted to analog video 
using an internal digital to analog converter (DAC) . The 
digital audio is converted to analog in the audio digital to 
analog converter (DAC) 465. The display engine 450 prepares 
the images for display on a display screen. 

[0036] The display engine 450 lags behind the MPEG video 
decoder 445 by a variable delay time. Because the display 
engine 4 50 lags behind the MPEG video decoder 44 5, the 
decoded images are buffered in image buffers 425a for 
display by the display engine 450. In order for the display 
engine 450 to accomplish its objective of being able to 
present the decoded images at their correct intended 
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presentation time, the display engine 450 uses various 
parameters decoded by the MPEG video decoder 445. 

[0037] The parameters can include, for example, but are not 
limited to, picture structure indicator 350a, progressive 
flag 350b, presentation time stamp (PTS) present flag 
350c, progressive frame flag 350d, picture structure 
indicator 350e, PTS 350f, pan-scan vectors 350g, aspect 
ratio 350h, decode and display horizontal size parameter 
350i, decode and display vertical size parameter 350j, top 
field first parameter 350k, repeat first field parameter 
3501 progressive sequence parameter 360a, , or frame rate 
parameter 360b. The parameters can also include parameters 
that are function of the aforementioned parameters, such 
as, but not limited to still picture interpolation mode, 
motion picture interpolation mode. 

[0038] Accordingly, the MPEG video decoder 445 stores the 
parameters associated with a particular image in a 
parameter buffer 425b associated with the image buffer 425a 
storing the particular decoded image. The display engine 
450 retrieves the parameters needed for displaying the 
images on the display device and displays the images on the 
display device based on the parameters. 

[0039] Referring now to FIGURE 5, there is illustrated a flow 
diagram describing the decoding and displaying of an image 
in accordance with an embodiment of the present invention. 
At 505, the MPEG video decoder 445 receives the MPEG video 
elementary stream comprising encoded compressed images and 
encoded parameters. At 510, MPEG video decoder 445 decodes 
the image and parameters associated therewith. The decoded 
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image is buffered in an image buffer 425a (at 515) and the 
parameters are stored (at 52 0) in a parameter buffer 42 5b 
associated with the image buffer 425a. 

[0040] The parameters can include, for example, but are not 
limited to, still picture interpolation mode 350a, motion 
picture interpolation mode 350b, presentation time stamp 
(PTS) present flag 350c, progressive frame flag 350d, 
picture structure indicator 350e, PTS 350f, pan-scan 
vectors 350g, aspect ratio 350h, decode and display 
horizontal size parameter 350i, decode and display vertical 
size parameter 350j , progressive sequence parameter 360a, 
top field first parameter 360b, repeat first field 
parameter 360c, or frame rate parameter 360d. 

[0041] At 525, the display engine 450 retrieves the 
parameters stored in the parameter buffer 425b. The display 
engine uses the parameters retrieved during 52 5 to display 
the decoded image (530) . For example, if the display engine 
450 detects a change in the parameters of two successive 
decoded images sent out for display, the display engine 450 
can initiate a display hardware reconfiguration before the 
decoded image with new display parameters is sent to 
screen. 

[0042] The decoder system as described herein may be 
implemented as a board level product, as a single chip, 
application specific integrated circuit (ASIC) , or with 
varying levels of the decoder system integrated on a single 
chip with other portions of the system as separate 
components. The degree of integration of the monitoring 
system will primarily be determined by speed of incoming 
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data, and cost considerations. Because of the sophisticated 
nature of modern processors, it is possible to utilize a 
commercially available processor, which may be implemented 
external to an ASIC implementation of the present system. 
Alternatively, if the processor is available as an ASIC 
core or logic block, then the commercially available 
processor can be implemented as part of an ASIC device 
wherein the flow charts described herein can be implemented 
as instructions in firmware. 

[0043] While the invention has been described with reference 
to certain embodiments, it will be understood by those 
skilled in the art that various changes may be made and 
equivalents may be substituted without departing from the 
scope of the invention. In addition, many modifications 
may be made to adapt particular situation or material to 
the teachings of the invention without departing from its 
scope. Therefore, it is intended that the invention not be 
limited to the particular embodiment (s) disclosed, but that 
the invention will include all embodiments falling within 
the scope of the appended claims. 
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